| Task: Create Test Specifications (AST) |
|
Purpose
| The creation of the test specifications per test unit. |
Relationships
| Roles | Primary:
| Additional:
| Assisting:
|
| Outputs |
|
Main Description
Method of operation
The testers specify the necessary tests for the test units in the test plan. After completion, the test
specifications are placed under configuration management. Depending on the test type and test technique selected for
the test unit, this activity may consist of anything from the creation of a checklist to the design and specification
of test cases according to a test design technique or to the design of a test with other techniques. The possibilities
are further explained below. Explanations are also given of a scalable regression test and of the relationship between
this phase and exploratory testing.
In the course of this activity, problems may arise with the test basis. Roughly, these can be categorised as follows:
-
Defects - As with the testability review, the testers may find shortcomings and/or ambiguities in the test basis.
The testers create a defect report on this. Via the defects procedure, it is passed to the test basis supplier, who
can then solve it.
-
Absence of test basis - If the testability review has been insufficiently executed, it may only appear at this
stage that certain parts of the test basis are missing or not detailed enough, so that they are not, or not
sufficiently, testable. The same types of measures as adopted with the testability review may be considered.
With iterative or agile system development, the test basis is often not 100% complete at the start of the
iteration, but is completed during the iteration. Besides the above mentioned measures, it is advisable to carry
out a minimal testability review with each addition to the test basis before specifying tests based on the
addition.
-
Unstable test basis - If the supplier of the test basis makes regular changes to it, for example because of
defects or change proposals, this makes for an unstable test basis. With every change, the testers have to examine
the relevant test specifications to see whether adjustments are necessary. These reworking operations are always
difficult to estimate in advance. The test manager is well advised to arrange a certain level of reserve budget and
time for this when creating the test plan. If these are exceeded, the project management should be notifi ed that
more time and finances are required. Other possible measures are to defer the specifying of the tests for the
unstable parts in the plan or to create the logical test cases, but to delay the physical makeup of them until the
test basis is (more) stable.
Test design techniques
Test design techniques are employed in the creation of the test cases. They enable the testers to derive test
cases that achieve a particular coverage from a test basis in a clear, transferable and reproducible manner. A test case
consists of a description of the initial situation, the test action and the predicted result. The specifying of test cases
according to a test design technique follows the following general steps:
-
1. Identifying test situations
-
2. Creating logical test cases
-
3. Creating physical test cases
-
4. Establishing the starting point
-
5. Creating the test script.
The concepts in these steps are explained in more detail in section Essential Test Design Concepts, including examples. Further explanation of the steps
themselves is given in "Steps" section below.
A choice has to be made in the establishment of the result of these steps, which collectively form the test
specification. Generally, two options are possible. The first is to put everything together in one file, usually a Word
document or spreadsheet. The second is to make a split in the design steps (identifying test situations and creating
logical test cases) and the steps required for test execution (the rest, resulting in the script). The advantage of the
first is that maintenance on the test cases is made easier, since everything is together. The second option does
not have this advantage, but does have two others. The testing operative (e.g. a user who has to carry out the test but
who has not designed it) is not distracted by the design steps and only sees the information that is necessary to carry
out the test. The second advantage is that the information required for the execution can be documented in a form other
than word processing or spreadsheet, such as a testware management tool or a database or directly in a tool for
automated test execution.
Checklists
Besides the specifying of test cases, many tests take place with the aid of checklists. These are used with
simple functional tests, but also for the static testing of e.g. maintainability, manageability, user-friendliness or
security. While a checklist is usually specific to the situation, testers often use a general checklist as a basis and
make specific adjustments to this. The general checklists may be supplied from within the organisation (by the test
department) or from the literature or via the Internet. Various examples of checklists for testing certain quality
characteristics can be found at www.tmap.net. The creation (and
execution) of a checklist requires a competent tester with the necessary knowledge of the object part or characteristic
under test. It is therefore advisable to have the checklist reviewed.
Other techniques
Apart from the specifying of test cases according to test design techniques and the creation of checklists,
other techniques are possible that do not fall into either of the above categories. These techniques mainly apply to
the testing of quality characteristics, such as portability, usability, performance and security.
Products
-
Test basis defects
-
Test specifications (checklists, test cases, test scripts)
Techniques
Tools
-
Defect management tool
-
Test design tool
-
Model-based testing tool
-
Testware management tool
-
Automated test execution tool
-
Performance, load and stress test tool.
|
Steps
|
1. Identifying test situations
Each test design technique is aimed at achieving a certain coverage in order to find particular types of defects. In this,
the first step is to distinguish the situations in the test basis that are to be tested. The test basis consists of, for
example, the system requirements, the functional design, the user manual and/or the administrative procedures. This test
basis is gone through and each situation to be tested is identified in it. The test design technique prescribes, as
regards situations to be tested, for example that the true and false situation is tested in respect of every
condition, or that each input field is tested with a valid and invalid input value. Since the distinguishing of test
situations is different for every test design technique, this step contains the biggest differences between the various
test design techniques.
|
2. Creating logical test cases
In principle, a test case goes through the relevant part of the test object (e.g. a function) from beginning to end,
covering one or more test situations in doing so.
The logical test cases clearly demonstrate:
-
What the test situations are
-
That all the defi ned test situations have been covered by test cases.
The creation of logical test cases may be skipped if the test situations can be directly executed and assessed one by one. |
3. Creating physical test cases
In practical terms, the physical test case describes what needs to be done, in which the three basic elements of a test
case are recognisable: initial situation, action and predicted result. In the physical makeup, each situation to be
tested, or logical test case, is completed with as much detail as possible, so that later, during the test execution,
the work can be performed as efficiently as possible. Obviously, the choice of physical makeup is based on the
preceding steps, but it is also influenced by agreements on, for example, the use of a certain (central) starting
point, e.g. a (blind) copy of the production database. It should also be agreed that the various tests should not be
able to disrupt each other during the execution, for example because different test cases use the same data
(competition). If one test case is “Remove order lines” while the other is “Change order lines”, there is a strong
chance that the predicted and justifi able results will not be realised. A solution is, for example, to agree
particular series of numbers for the order rules for the sake of the testing of various functions. In this connection,
it is also advisable to create no dependencies between test cases. If the execution of one test case is conditional for
the execution of another, the failure of the first test case will also mean that the second test case cannot be
executed. Only when the defect that is blocking the first test case is solved can the second test case be executed. If
this again results in a defect found, it can lead to extra and unnecessary reworking operations.
Section “Test situation, test case and test script” in Essential Test Design Concepts contains a number of considerations as to how
far the tester should go in the physical makeup of test cases.
|
4. Establishing the starting point
To be able to execute a test case, an initial situation is necessary. Initial situations often contain the same data for
several test cases. Such data are therefore included in a so-called starting point for the entire test and not separated
for each test case. This starting point is made ready prior to the test execution. A step further is where the starting
points for various tests may also show a (big) overlap. For that reason, it is often a case of one or more central starting
points applying to several tests. This is described extensively in Define Central Starting Points (AST).
|
5. Creating the test script
As a last step, one or more test scripts are produced. In this, the test actions and checks of the physical test cases are
described in a sequence that best suits test execution. In this, the test cases should not be able to disrupt each other.
The test script as such is the stepped plan for the test execution and also offers the possibility of progress monitoring.
The physical test cases and the starting points naturally form the basis for the creation of the test script.
Considerations for combining test cases in a single script are described in section "Test situation, test case and test
script" of Essential Test Design Concepts.
The generic content list of a script is as follows:
-
Unique identity, consisting of:
-
Version
-
Creator
-
Test basis, including version.
-
Preparing the starting point - For example, by setting the system date, restoring a particular backup and adding
particular test data
-
Test actions and verification - The physical test cases in a sequence suitable for execution, with the necessary
initial situation for each test case, action and result verification. When an appropriate starting point is
established, nothing more is usually required to be done in respect of the initial situation.
-
Cleaning up the environment - Ensure that the results of the executed test are cleaned up if necessary so that they
do not get in the way of other testers (remember, for example, to restore the system date).
The above says nothing about how the script is stored, for example in a Word document, spreadsheet, automated test or a
database. Section "Test situation, test case and test script" of Essential Test Design Concepts supplies a number of possibilities and
considerations.
In some cases, the support of tools is possible, for example, tools for generating test data for deriving test cases, or
even model-based test tools (see Types Of Test Tools). |
|
More Information
Copyright
© 2006, Sogeti Nederland B.V. All rights
reserved.
|
|
